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Abstract 

The classical zero-one law for first-order logic on random graphs says that for every 
first-order property Lp in the theory of graphs and every p £ (0, 1), the probability that 
the random graph G{n,p) satisfies ip approaches either or 1 as n approaches infinity. 
It is well known that this law fails to hold for any formalism that can express the parity 
quantifier: for certain properties, the probability that G{n,p) satisfies the property need 
not converge, and for others the limit may be strictly between and 1. 

In this work, we capture the limiting behavior of properties definable in first order 
logic augmented with the parity quantifier, F0[®], over G{n,p), thus eluding the above 
hurdles. Specifically, we establish the following "modular convergence law" : 

For every F0[©] sentence (p, there are two explicitly computable rational 
numbers oq, ai, such that for i G {0, 1}, as n approaches infinity, the prob- 
ability that the random graph G{2n + i,p) satisfies ip approaches a^. 

Our results also extend appropriately to FO equipped with Modq quantifiers for prime 

q- 

In the process of deriving the above theorem, we explore a new question that may be 
of interest in its own right. Specifically, we study the joint distribution of the subgraph 
statistics modulo 2 of G{n,p): namely, the number of copies, mod 2, of a fixed number 
of graphs Fi, . . . ,Fi of bounded size in G{n,p). We first show that every F0[©] property 
if is almost surely determined by subgraph statistics modulo 2 of the above type. Next, 
we show that the limiting joint distribution of the subgraph statistics modulo 2 depends 
only on n mod 2, and we determine this limiting distribution completely. Interestingly, 
both these steps are based on a common technique using multivariate polynomials over 
finite fields and, in particular, on a new generalization of the Gowers norm. 

The first step above is analogous to the Razborov-Smolensky method for lower 
bounds for ACO with parity gates, yet stronger in certain ways. For instance, it allows 
us to obtain examples of simple graph properties that are exponentially uncorrelated 
with every F0[©] sentence, which is something that is not known for ACO[©]. 



*UC Santa Cruz and IBM Almaden Research Center, kolaitis@cs.ucsc.edu 
^Massachusetts Institute of Technology, swastik@mit.edu 



Contents 

1 Introduction 

1.1 Methods 

1.1.1 The distribution of subgraph frequencies mod q, polynomials and 
Gowers norms 

1.1.2 Quantifier elimination 

1.2 Comparison with ACO[©] 

2 The Modular Convergence Law 

2.1 Pseudorandomness against FO[0] 

3 The Distribution of Subgraph Frequencies mod q 

3.1 Preliminary lemmas 

3.2 Proof of the equidistribution theorem 

4 The Bias of Polynomials 

4.1 The //-Gowers norm 

5 Proof of Theorem [273] 

5.1 Labelled graphs and labelled subgraph frequencies 

5.2 The quantifier eliminating theorem 

6 Quantifier Elimination 

6.1 Counting extensions 

6.2 The distribution of labelled subgraph frequencies mod q 

6.3 Proof of Theorem 15.81 

7 Counting Extensions 

7.1 Subgraph frequency arithmetic 

7.2 Proof of Theorem 16.11 

8 The Distribution of Labelled Subgraph Frequencies mod q 

8.1 Equidistribution of labelled subgraph copies 

8.2 Proof of Theorem 



9 Concluding Remarks 



1 Introduction 



For quite a long time, combinatorialists have studied the asymptotic probabihties of prop- 
erties on classes of finite structures, such as graphs and partial orders. Assume that C is 
a class of finite structures and let Pr„, n > 1, be a sequence of probability measures on 
all structures in C with n elements in their domain. If Q is a property of some structures 
in C (that is, a decision problem on C), then the asymptotic probability Pr{Q) of Q on C 
is defined as Pr((5) = lim„^oo Pr„((5), provided this limit exists. In this paper, we will 
be focusing on the case when C is the class Q of all finite graphs, and Pr„ = G{n,p) for 
constant p; this is the probability distribution on n-vertex undirected graphs where be- 
tween each pair of nodes an edge appears with probability p, independently of other pairs 
of nodes. For example, for this case, the asymptotic probabilities Pr(CONNECTiviTY) = 1 
and Pr(HAMlLTONlClTY) = 1; in contrast, if Pr„ = G{n,p{n)) with p{n) = 1/n, then 
Pr(CONNECTIVITY) = and Pr(HAMILTONICITY) = 0. 

Instead of studying separately one property at a time, it is natural to consider formalisms 
for specifying properties of finite structures and to investigate the connection between the 
expressibility of a property in a certain formalism and its asymptotic probability. The 
first and most celebrated such connection was estabhshed by Glebskii et al. |GKLT69j and, 
independently, by Fagin |Fag76| , who showed that a. 0-1 law holds for first-order logicQ FO 
on the random graph G{n,p) with p a constant in (0, 1); this means that if Q is a property 
of graphs expressible in FO and Pr„ = G{n,p) with p a constant in (0,1), then Pr(Q) 
exists and is either or 1. This result became the catalyst for a series of investigations in 
several different directions. Specifically, one line of investigation [SS871 ISS88j investigated 
the existence of 0-1 laws for first-order logic FO on the random graph G{n,p{n)) with 
p(n) = < a < 1. Since first-order logic on finite graphs has limited expressive power 
(for example, FO cannot express Connectivity and 2-Colorability), a different line of 
investigation pursued 0-1 laws for extensions of first-order logic on the random graph G{n,p) 
with p a constant in (0, 1). In this vein, it was shown in [BG K851 IKV87j that the 0-1 law 
holds for extensions of FO with fixed-point operators, such as least fixed-point logic LFP, 
which can express Connectivity and 2-Colorability. As regards to higher-order logics, 
it is clear that the 0-1 law fails even for existential second-order logic ESO, since it is well 
known that ESO = NP on finite graphs |Fag74| . In fact, even the convergence law fails for 
ESO, that is, there are ESO-expressible properties Q of finite graphs such that Pi(Q) does 
not exist. For this reason, a separate line of investigation pursued 0-1 laws for syntactically- 
defined subclasses of NP. Eventually, this investigation produced a complete classification 
of the quantifier prefixes of ESO for which the 0-1 law holds |KV87l IKV901 IPS89j . and 
provided a unifying account for the asymptotic probabilities of such NP-complete problems 
as A;-COLORABiLiTY, A; > 3. 

Let L be a logic for which the 0-1 law (or even just the convergence law) holds on the random 
graph G{n,p) withp a constant in (0, 1). An immediate consequence of this is that L cannot 

^Recall that the formulas of first-order logic on graphs are obtained from atomic formulas E{x, y) (in- 
terpreted as the adjacency relation) and equality formulas x = y using Boolean combinations, existential 
quantification, and universal quantification; the quantifiers are interpreted as ranging over the set of vertices 
of the graph (and not over sets of vertices or sets of edges, etc.). 
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express any counting properties, such as Even Cardinality ("there is an even number 
of nodes"), since Pr2n(EvEN Cardinality) = 1 and Pr2n+i(EvEN Cardinality) = 0. 
In this paper, we turn the tables around and systematically investigate the asymptotic 
probabilities of properties expressible in extensions of FO with counting quantifiers Mod*, 
where g is a prime number. The most prominent such extension is F0[©], which is the 
extension of FO with the parity quantifier Mod2. The syntax of F0[©] augments the syn- 
tax of FO with the following formation rule: if (p{y) is a F0[©] -formula, then (Byf{y) 
is also a F0[©] -formula; this formula is true if the number of y's that satisfy <f{y) is 
odd (analogously, Modl^yip{y) is true if the number of y's that satisfy if{y) is congru- 
ent to i mod q). A typical property on graphs expressible in F0[©] (but not in FO) is 
V := {G : every vertex of G has odd degree}, since a graph is in V if and only if it satisfies 
the FO[©]-sentence Vx © yE{x, y). 

Our main result (see Theorem 12. ip is a modular convergence law for F0[©] on G{n,p) with 
p a constant in (0,1). This law asserts that if is a F0[©] -sentence, then there are two 
explicitly computable rational numbers ao, ai, such that, as n — > oo, the probability that 
the random graph G{2n + i,p) satisfies ip approaches aj, for i = 0,1. Moreover, ao and ai 
are computable and are of the form r/2^, where r and s are non- negative integers. We also 
establish that an analogous modular convergence law holds for every extension FO[Modg] 
of FO with the counting quantifiers {Mod^ : i € [q — 1]}, where g is a prime. It should be 
noted that results in [HKL96j imply that the modular convergence law for F0[©] does not 
generalize to extensions of F0[©] with fixed-point operators. This is in sharp contrast to 
the aforementioned 0-1 law for FO which carries over to extensions of FO with fixed-point 
operators. 

1.1 Methods 

Earlier 0-1 laws have been established by a combination of standard methods and tech- 
niques from mathematical logic and random graph theory. In particular, on the side of 
mathematical logic, the tools used include the compactness theorem, Ehrenfeucht-Fraisse 
games, and quantifier elimination. Here, we establish the modular convergence law by com- 
bining quantifier elimination with, interestingly, algebraic methods related to multivariate 
polynomials over finite fields. In what follows in this section, we present an overview of the 
methods and techniques that we will use. 

1.1.1 The distribution of subgraph frequencies mod q, polynomials and Gowers 
norms 

Let us briefly indicate the relevance of polynomials to the study of F0[©] on random graphs. 
A natural example of a statement in F0[©] is a formula such that G satisfies (p if and 
only if the number of copies H in G is odd, for some graph H (where by copy we mean 
an induced subgraph, for now). Thus understanding the asymptotic probability of ip on 
G{n,p) amounts to understanding the distribution of the number of copies (mod 2) of H 
in G{n,p). 
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In this spirit, we ask: what is the probability that in G(n, 1/2) there is an odd number 
of triangles (where we count unordered triplets of vertices {a, 6, c} such that a, 6, c are all 
pairwise adjacenlH)? 

We reformulate this question in terms of the following "triangle polynomial" , that takes the 
adjacency matrix of a graph as input and returns the parity of the number of triangles in 
the graph; Pa : {0, Ij'^a) — > {0, 1}, where 

{61,62,63} forming a A 

where the arithmetic is mod 2. Note that for the random graph G(n, 1 /2), each entry of the 
adjacency matrix is chosen independently and uniformly from {0, 1}. Thus the probability 
that a random graph G € G(n, 1/2) has an odd number of triangles is precisely equal 
to Pr^GZ^ [-Pa(2;) = 1]. Thus we have reduced our problem to studying the distribution 
of the evaluation of a certain polynomial at a random point, a topic of much study in 
pseudorandomness and algebraic coding theory, and we may now appeal to tools from these 
areas. 

In Section [3l via the above approach, we show that the probability that G(n, 1/2) has an 
odd number of triangles equals 1/2 it 2~^("'). Similarly, for any connected graph F 7^ Ki 
(the graph consisting of one vertex), the probability that G{n, 1/2) has an odd number of 
copies H of F is also 1/2 ± 2-^(") (when F = Ki, there is no randomness in the number of 
copies of F in G(n, 1/2)!). In fact, we show that for any collection of distinct connected 
graphs Fi, . . . , (7^ Ki), the joint distribution of the number of copies mod 2 of Fi, . . . , F^ 
in G(n, 1/2) is 2~^(")-close to the uniform distribution on Z2, i.e., the events that there are 
an odd number of Fj are essentially independent of one another. 

Generalizing the above to G{n,p) and counting mod q for arbitrary p £ (0, 1) and arbitrary 
integers q motivates the study of new kinds of questions about polynomials, that we believe 
are interesting in their own right. For G{n,p) with arbitrary p, we need to study the 
distribution of P{x), for certain polynomials P, when x G is distributed according to 
the p-biased measure. Even more interestingly, for the study of FO[Modg], where we are 
interested in the distribution of the number of triangles mod q, one needs to understand 
the distribution of P(x) (P is now a polynomial over Zg) where x is chosen uniformly 
from {0, 1}™" C (as opposed to x being chosen uniformly from all of Z™, which is 
traditionally studied) . In Section HI we develop all the relevant polynomial machinery in 
order to answer these questions. This involves generalizing some classical results of Babai, 
Nisan and Szegedy [BNS89] on correlations of polynomials. The key technical innovation 
here is our definition of a //-Gowers norm (where /i is a measure on Z™) that measures 
the correlation, under of a given function with low-degree polynomials (letting fj, be the 
uniform measure, we recover the standard Gowers norm). After generalizing several results 
about the standard Gowers norm to the ;U-Gowers norm case, we can then use a technique 
of Viola and Wigderson |VW07j to establish the generalization of |BNS89j that we need. 

■^Counting the number of unordered triples is not expressible in F0[©], we ask this question only for 
expository purposes (nevertheless, we do give an answer to this question in Section |3]). 
^ with a certain precise definition of "copy". 
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1.1.2 Quantifier elimination 



Although we studied the distribution of subgraph frequencies mod q as an attempt to 
determine the limiting behavior of only a special family of FO[Modg] properties, it turns out 
that this case, along with the techniques developed to handle it, play a central role in the 
proof of the full modular convergence law. In fact, we reduce the modular convergence law 
for general FO[l\/lodg] properties to the above case. We show that for any FO[Modg] sentence 
99, with high probability over G G G{n,p), the truth of 99 on G is determined by the number 
of copies in G, mod q, of each small subgraph. Then by the results described earlier on the 
equidistribution of these numbers (except for the number of Ki, which depends only on n 
mod q), the full modular convergence law for FO[Modq] follows. 

In SectionlH we establish such a reduction using the method of elimination of quantifiers. To 
execute this, we need to analyze FO[Modq] formulas which may contain free variables (i.e., 
not every variable used is quantified). Specifically, we show that for every FO[Modg] formula 
(/5(q!i, . . . , Ofc), with high probability over G G G{n,p), it holds that for all vertices wi, . . . ,Wk 
of G, the truth of ip^wi, . . . ,Wk) is entirely determined by the following data: (a) which of 
the WijWj pairs are adjacent, (b) which of the Wi, Wj pairs are equal to one another, and 
(c) the number of copies "rooted" at wi, . . . ,Wk, mod q, of each small labelled graph. This 
statement is a generalization of what we needed to prove, but lends itself to inductive proof 
(this is quantifier elimination). This leads us to studying the distribution (via the polynomial 
approach described earlier) of the number of copies of labelled graphs in G; questions of 
the form, given two specified vertices v,w (the "roots"), what is the probability that there 
are an odd number of paths of length 4 in G € G{n,p) from v to w? After developing the 
necessary results on the distribution of labelled subgraph frequencies, combined with some 
elementary combinatorics, we can eliminate quantifiers and thus complete the proof of the 
modular convergence law. 

1.2 Comparison with ACO[©] 

Every F0[©] property naturally defines a family of boolean functions /„ : {0, Ij^a) —)■ {0, 1}, 
such that a graph G satisfies (p if and only if /„(j4g) = 1, where Aq is the adjacency matrix 
of G. This family of functions is easily seen to be contained in ACO[©], which is ACO with 
parity gates (each V becomes an AND gate, 3 becomes a OR gate and © becomes a parity 
gate). This may be summarized by saying that F0[©] is a highly uniform version of ACO[©]. 

Currently, all our understanding of the power of ACO[©] comes from the Razborov-Smolensky 
[Raz87t rSmo87j approach to proving circuit lower bounds on ACO[©]. At the heart of this 
approach is the result that for every ACO[©] function /, there is a low-degree polynomial P 
such that for 1 — e(n) fraction of inputs, the evaluations of / and P are equal. Note that 
this result automatically holds for F0[©] (since F0[©] C ACO[©]). 



We show that for the special case when / : {0, Ijvai — >. {0, 1} comes from an F0[©] property 
if, a significantly improved approximation may be obtained: (i) We show that the degree of 
P may be chosen to be a constant depending only on ip, whereas the Razborov-Smolensky 
approximation required P to be of polylog(n) degree, (ii) The error parameter e{n) may be 
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chosen to be exponentially small in n, whereas the Razborov-Smolensky method only yields 
e(n) = 2^^°s°'^'", (Hi): Finally, the polynomial P can be chosen to be symmetric under 
the action of Sn on the (2) coordinates, while in general, the polynomial produced by the 
Razborov-Smolensky approach need not be symmetric (due to the randomness involved in 
the choices). 

These strengthened approximation results allow us, using known results about pseudoran- 
domness against low-degree polynomials, to show that (i) there exist explicit pseudorandom 
generators that fool F0[©] sentences, and (ii) there exist explicit functions / such that for 
any F0[©] formula ip, the probability over G G G{n,p) that /(G) = f{G) is at most 

1 _|_ ^ The first result follows from the pseudorandom generators against low-degree 
polynomials due to Bogdanov- Viola [BV07j . Lovett |Lov08j and Viola |Vio08j . The second 
result follows from the result of Babai, Nisan and Szegedy [BNS89j . and our generalization 
of it, giving explicit functions that are uncorrelated with low degree polynomials. 

Obtaining similar results for ACO[©] is one of the primary goals of modern day "low-level" 
complexity theory. 

Organization of this paper: In the next section, we formally state our main results and 
some of its corollaries. In Section [3l we determine the distribution of subgraph frequencies 
mod q. In Section HI we introduce the /i-Gowers Norm use it to prove some technical results 
on the bias of polynomials needed for the previous section. In Section O we state the 
theorem which implements the quantifier elimination and describe the plan for its proof. 
This plan is then executed in Sections EJ [7] and El We conclude with some open questions. 

2 The Modular Convergence Law 

We now state our main theorem. 

Theorem 2.1 Let q be a prime. Then for every FO[Modg] -sentence (p, there exist rationals 
oo, . . . , flq-i such that for every p G (0, 1) and every z G {0, 1, . . . , g — 1}, 

lim Pr \G satisfies 09I = Oj. 

n^oo G£G(n v) 
n=i mod q ^ 

Remark The proof of Theorem 12.11 also yields: 

• Given the formula if, the numbers ao, . . . , a^-i can be computed. 

• Each is of the form r/g^, where r, s are nonnegative integers. 

• For every sequence of numbers &01 • • • > ^g-i ^ [0, 1], each of the form r/q^, there is a 
FO[Modg]-sentence such that for each i, the number a, given by the theorem equals 
hi. 
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Before we describe the main steps in the proof of Theorem 12.11 we make a few definitions. 

For graphs F = {Vf,Ef) and G = (VcEg), an (injective) homomorphism from F to G is 
an (injective) map X '■ ^ Vg that maps edges to edges, i.e., for any (n, v) G Ep, we have 
{x{u),xiv)) € Eg- Note that we do not require that x maps non-edges to non-edges. We 
denote by [-F](G) the number of injective homomorphisms from F to G, and we denote by 
[F]q(G) this number mod q. We let aut(-F) := be the number of automorphisms of 

F. 

The following lemma (which follows from Lemma 16.51 in Section [6]) , shows that for some 
graphs F, as G varies, the number [F](G) cannot be arbitrary. 

Lemma 2.2 Let F be a connected graph and G be any graph. Then aut(-F) | [-F](G). 

For the rest of this section, let q be a fixed prime. Let Conn" be the set of connected graphs 
on at most a vertices. For any graph G, let the subgraph frequency vector freq^ G 
be the vector such that its value in coordinate F (F £ Conn'*) equals [F]q{G), the number 
of injective homomorphisms from F to G mod q. Let FFreq(a), the set oi feasible frequency 
vectors, be the subset of 2^°""" consisting of all / such that for all F G Conn", fp G 
aut(F) • Zq := {aut(F) ■ x \ x e Zg}. By Lemma \T2\ for every G and a, freq^ G FFreq(a), 
i.e., the subgraph frequency vector is always feasible. 

We can now state the two main technical results that underlie Theorem 12.11 

The first states that on almost all graphs G, every FO[Moclg] formula can be expressed in 
terms of the subgraph frequencies, [F]q{G), over all small connected graphs F. 

Theorem 2.3 (Subgraph frequencies mod q determine FO[Modg] formulae) For ev- 
ery fO[Modq]-sentence ip of quantifier depth t, there exists an integer c = c{t,q) and a 
function : Z^°""" {0, 1} such that for all p G (0, 1), 

Pr [{G satisfies (p) <^ (-(/^(freq^;) = !)]>!- exp(-n). 

G&G[n,p) 

This result is complemented by the following result, that shows the distribution of subgraph 
frequencies mod g in a random graph G G G{n,p) is essentially uniform in the space of all 
feasible frequency vectors, up to the obvious restriction that the number of vertices (namely 
the frequency of Ki in G) should equal n mod q. 

Theorem 2.4 (Distribution of subgraph frequencies mod q depends only on n mod q) 

Let p G (0,1). Let G G G{n,p). Then for any constant a, the distribution o/ freq^. is 
exp{—n)-close to the uniform distribution over the set 

{/ G FFreq(a) : fx^ = n mod q}. 

Theorem 12.41 is proved in Section [3] by studying the bias of multivariate polynomials over 
finite fields via a generalization of the Gowers norm. Theorem 12.31 is proved in Section [6] 
using two main ingredients: 
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1. A generalization of Theorem 12.41 that determines the joint distribution of the frequen- 
cies of "labelled subgraphs" with given roots (see Section [8]). 

2. A variant of quantifier elimination (that may be called quantifier conversion) designed 
to handle Modg quantifiers that crucially uses the probabilistic input from the previous 
ingredient (see Section [6|). 

Proof of Theorem I2.lt Follows by combining Theorem 12.31 and Theorem 12. 4[ □ 



2.1 Pseudorandomness against F0[©] 

We now point out three simple corollaries of our study of F0[©] on random graphs. 

Corollary 2.5 (FO[l\/lodg] is well approximated by low-degree polynomials) For ev- 
ery FO[Modg] -sentence (p, there is a constant d, such that for each n € N, there is a degree 
d polynomial P{{Xe)^^^n^^) E Zg[(Xe)gg^n-j], such that for all p G (0, 1), 

Pr [{G satisfies <f) ^ P{Ag) = 1] > 1 - 2-^("), 
where Aq € {0, Ij^a) is the adjacency matrix ofG. 

Proof Follows from Theorem 12.31 and the observation that for any graph F of constant 
size, there is a polynomial Q{{Xe)^^(ri\^) of constant degree, such that Q{Ag) = [F]q{G) for 
all graphs G. □ 



Corollary 2.6 (PRGs against F0[©]) For each s G N and constant e > 0, there is a 
constant c > such that for each n, there is a family T of Q{n^) graphs on n vertices, 
computable in time poly(n^), such that for all FO[0] -sentences (p of size at most s, and for 
allpe (0,1), 

I Pr \G satisfies ip] — Pr \G satisfies (pi I < e. 
'ce^^ GeG(n,py 

Proof For p = 1/2, this follows from the previous corollary and the result of Viola |Vio08] 
(building on results of Bogdanov- Viola |BV07j and Lovett |Lov08j ) constructing a pseu- 
dorandom generator fooling low-degree polynomials under the uniform distribution. For 
general p, note that the same family from the p = 1/2 case works, since the distribution 
of subgraph frequencies given in Theorem 12.41 is independent of p. □ 

The analogue of the previous corollary for FO was proved in |GS7H IBEH81] (see also [BROSj 

EnTM]). 
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Corollary 2.7 (Explicit functions exponentially hard for F0[©]) There is an explicit 
function f : {0, 1}(2) {0, 1} such that for every F0[©] -sentence ip, 

Pr [{G satisfies ip) ^ {fiAc) = !)]<! + 2"^^. 

Proof Follows from Corollary 12.51 and the result of Babai, Nisan, Szegedy |BNS89] 
(for p = 1/2) and its generalization, Lemma 14.11 (for general p), constructing functions 
exponentially uncorrelated with low degree polynomials under the p-biased measure. It 
actually follows from our proofs that, one may even choose a function / that is a graph 
property (namely, invariant under the action of Sn on the coordinates). □ 



3 The Distribution of Subgraph Frequencies mod q 

In this section, we prove Theorem l2.4l on the distribution of subgraph frequencies in G{n,p). 

We first make a few definitions. If is a connected graph and G is any graph, a copy of F 
in G is a set E C Eg such that there exists an injective homomorphism x from F to G such 
that E = xi^p) ■= {(x(^);X(^)) I iv,w) G Ep}. We denote the set of copies of F in G 
by Cop(F, G), the cardinality of Cop(F, G) by {F){G), and this number mod q by {F)q{G). 
We have the following basic relation (which follows from Lemma 16.51 in Section [6]) . 

Lemma 3.1 If F is a connected graph with \Ep\ > 1, then 

[F](G) = aut(F).(F)(G). 

For notational convenience, we view G{n,p) as a graph whose vertex set is [n] and whose 
edge set is a subset of (^2')- 

We can now state the general equidistribution theorem from which Theorem 12.41 will follow 
easily (We use the notation Qq^p^d{n) to denote the expression 0(n), where the implied 
constant depends only on q, p and d). Note that this theorem holds for arbitrary integers 
q, not necessarily prime. 

Theorem 3.2 (Equidistribution of graph copies) Let q > I be an integer and let p G 
(0, 1). Let El, . . . ,Fi € Conn" be distinct graphs with 1 < \Ep^ \ < d. 

Let G G G{n,p). Then the distribution of ((Fi),(G), . . . , {Ee)q{G)) on is 2-f^^.p.d(")+^- 
close to uniform in statistical distance. 

Using this theorem, we complete the proof of Theorem 12. 4[ 

Proof of Theorem 12. 4t Let Ei,... ,Ei be an enumeration of the elements of Conn" 
except for Ki. By Theorem 13.21 the distribution of g = {{Ei)q{G)l_^ is 2~^("') close to 
uniform over Z^. Given the vector g, we may compute the vector freq^ by: 
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• (freqc')Ki = mod q. 

• For F G Conn'' \ {Ki}, {freq'^)F = Qf ■ aut(F) (by LemmaEH). 

This implies that the distribution of freq^ is 2~^''"''*-close to uniformly distributed over 
{/ G FFreq(a) : /k^ = n mod q}. □ 

Towards proving Theorem 13.2^ we now introduce some tools. 
3.1 Preliminary lemmas 

As indicated in the introduction, the distribution of subgraph frequencies is most naturally 
studied via the distribution of values of certain polynomials. The following lemma, which is 
used in the proof of Theorem l3.2l (and again in Section [5] to study the distribution of labelled 
subgraph frequencies), gives a simple sufficient criterion for the distribution of values of a 
polynomial to be "unbiased" . The proof appears in Section [H 

Lemma 3.3 Let q > 1 be an integer and let p G (0, 1). Xe0 C 2[™]. Let d > be an 
integer. Let Q{Zi, . . . , Z^) G . . . , Zm] be a polynomial of the form 

where deg{Q') < d. Suppose there exist £ = {i^i, . . . , Ej.} ^ !F such that: 

• \Ej\ = d for each j, 

• 7^ for each j . 

• Ej n Eji = for each j,j', 

• For each S eJ'XS, \S Ci {UjEj)\ < d. 

Let z = (zi, . . . , Zm) G be the random variable where, independently for each i, we have 
Pr[zj = V\ = p and Pr[zj = 0] = 1 — p. Then, 



^Q(z) 



E 

where to £ C is a primitive q^^-root of unity. 

The lemma below is a useful tool for showing that a distribution on is close to uniform. 

Lemma 3.4 (Vazirani XOR lemma) Let q > 1 be an integer and let lo C be a prim- 
itive q^^-root of unity. Let X = {Xi, . . . ,X^) be a random variable over Z^. Suppose that 
for every nonzero c G Z^, 



E 



< e. 



Then X is q^ ■ e-close to uniformly distributed over Z^, 
5* is a set, we use the notation 2^ to denote its power set. 
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3.2 Proof of the equidistribution theorem 



Proof of Theorem 13. 2t By the Vazirani XOR Lemma (Lemma 13. 4p . it suffices to show 
that for each nonzero c € Z^, we have |E [liJ-^] | < 2~^9.p,dW^ where R := X]iG[£] Cj(-Fi)g(G), 
and a; G C is a primitive q^^-root of unity. 

We win show this by appeahng to Lemma 13.31 Let m = (2). Let z € {0, IjCa') be the 
random variable where, for each e S ('2^); Zf. = 1 \f and only if e is present in G. Thus, 
independently for each e, Pr[2;e = 1] = P- 

We may now express R in terms of the Ze- Let K„ denote the complete graph on the vertex 
set [n]. Thus Cop(-Fj,K„) is the set of E that could potentially arise as copies of Fi in G. 
Then we may write, 

i?= j^c,(F,),(G)= j^c, Yi n^- 

iGM EeCop(Fi,K„)e€E 

= n 

where C 2'^'2') is the set |J..^^_^q Cop(Fj, K„), and for E & ce = Ci for the unique i 
satisfying E G Cop(-Fj,K„) (note that since the Fi are nonisomorphic connected graphs, the 
Cop(F„K n) are pairwise disjoint). 

Let Q(Z) e Zg[Z], where Z = (Ze)gg^[,i]-j be the polynomial Yl,EeT^EW(.,zE ^e- Then 
R = Q{z). We wish to show that 



E 



< 2~^'J,P,<i("). (1) 



We do this by demonstrating that the polynomial (5(Z) satisfies the hypotheses of Lemma[3]3j 

Let d* = maxj-c-^o I-F-fJ- Let iq S [i] be such that Ci^ ^ and I^Fipl = d* . Let 
X1jX2i ■ ■ ■ ,Xr G lnj(Fjp,Kn) be a collection of homomorphisms such that for all distinct 
G [r], we have Xj(^Fi(j) H Xj'(^f,q) = 0. Such a collection can be chosen greedily so 
that r = J^(§). Let E'j € Cop(FjQ,K„) be given by Xj{EFig)- Let f be the family of sets 
{El, . . . , Er} . We observe the following properties of the Ef 

1. For each j € [r], lE'jl = d* (since Xj is injective). 

2. For each j G [r], c^;^. = 7^ 0. 

3. For distinct j,/ G [r], £'j n £'j' = (by choice of the Xj)- 

4. For every S £ J^\£, \S (1 {[JjEj)\ < d*. To see this, take any S £ J^\£ and suppose 
\S n {(JjEj)\ > d*. Let i' G [i] be such that 7^ and 5 G Cop(Fj/,K„). Let 
X G lnj(Fj/,K„) with x{Ee.,) = S. By choice of d*, we know that |5| < d* . Therefore, 
the only way that \Sr\{LljEj)\ can be > d* is if (1) \S\ = d* , and (2) Sr\{[JjEj) = S, or 
in other words, S C (UjEj). However, since the Xi(^iy) are all pairwise disjoint, this 
implies that S C Ej for some j. But since \Ej\ = \S\, we have S = Ej, contradicting 
our choice of S. Therefore, \S fl ([JjEj)\ < d* for any S £ J^\£. 
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It now follows that T and E satisfy the hypothesis of Lemma 13. 3[ Consequently, 

(recalling that r = ^l{n/d) and d* < d) Equation ([1]) follows, completing the proof of the 
theorem. □ 



Remark We just determined the joint distribution of the number of injective homomor- 
phisms, mod q, from all small connected graphs to G{n,p). This information can be used in 
conjunction with Lemma 16.21 to determine the joint distribution of the number of injective 
homomorphisms, mod q, from all small graphs to G{n,p). 



4 The Bias of Polynomials 



Our main goal in this section is to give a full proof of Lemma 13.31 which gives a criterion 
for a polynomial to be unbiased. Along the way, we will introduce the /u-Gowers norm and 
some of its useful properties. 

Our proof of Lemma 13.31 will go through the following lemma (which is proved in the next 
subsection). It shows that "Generalized Inner Product" polynomials are uncorrelated with 
polynomials of lower degree. This generalizes a result of Babai Nisan and Szegedy |BNS89] 
(which dealt with the case q = 2 and p = 1/2). 



Lemma 4.1 Let q > 1 be an integer and let p E (0, 1). Let Ei 



subsets of [m] each of cardinality d. Let Q{Zi, 
the form 



^aj n Zi] +i?(Z), 



where each aj ^ and deg(i?(Z)) < d. Let z 
where, independently for each i, we have Pr[2;, 



Er be pairwise disjoint 
be a polynomial of 



1 Zm 



(zi, . . . , Zm) € be the random variable 



1] = p and VT[zi 



0] = 1 — p. Then, 



Given Lemma l4.ll we may now prove Lemma 13.31 

Proof of Lemma Let U = U^=i£^j. Fix any x G {0,1}H\C^, Qa;(Y) G 

Z(j[(^i)jeC/] be the polynomial 

E«M n I ( n ^0 +^'("^''^) 

s^r \jesn{[m]\u) / \iGSnu J 

so that Qx{y) = Q{x,y) for each y G Z^. Notice that the degree (in Y) of the term 
corresponding to G is IS*!"! By assumption, unless S = Ej for some j, we must have 

\snu\< d. 
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Therefore the polynomial QxO^) is of the form: 



Y,aE,J{Y, + R{Y), 

j=l i£Ei 



where deg(i?(Y)) < d. By Lemma |4. 11 



E 



CO 



QAy) 



< 



where y € {0, 1}^ with each yi = 1 independently with probability p. 

As Qxiy) = Qix,y), we get 



E 



UJ 



< 



where z^' S Zg is the random variable z conditioned on the event Zj = Xj for every j € [m]\[/. 
Now, the distribution of z is a convex combination of the distributions of z^ as x varies over 
{0, l}H\t^. This allows us to deduce that 



E 



UJ 



Q(z) 



as desired. 



□ 



4.1 The yU-Gowers norm 



The proof of Lemma l4.ll will use a variant of the Gowers norms. Let Q : Z™ TLq be any 
function, and define / : ^ C by f(x) = lo'^^^^ . The Gowers norm of / is an analytic 
quantity that measures how well Q correlates with degree d polynomials: the correlation of 
Q with polynomials of degree d—\ under the uniform distribution is bounded from above 
by the (i*'^-Gowers norm of /. Thus to show that a certain Q is uncorrelated with all degree 
d — \ polynomials under the uniform distribution, it suffices to bound the (i*'^-Gowers norm 
of /. In Lemma 14. H we wish to show that a certain Q is uncorrelated with all degree 
d — \ polynomials under a distribution [x that need not be uniform. To this end, we define 
a variant of the Gowers norm, which we call the /^-Gowers norm, and show that if the 
(d, ^)'''^-Gowers norm of / is small, then Q is uncorrelated with all degree d— \ polynomials 
under fi. We then complete the proof of Lemma l4. II by bounding the {d, fiY^-Goweis norm 
of the relevant /. 

We first define the //-Gowers norm and develop some of its basic properties. 

Let H be an abelian group and let /i be a probability distribution on H. For each d > 0, 
define a probability distribution fi^^^ on H'^^^ inductively by and, for d > 1, let 

IJ,^'^\x,ti, ...,td) equal 

/x^'^-^) (x, ^1, . . . , td^l) ■ fi^'^-^^ {x + td,tl,..., td-l) 
E,eHf^^''''Kz,ti,...,td-i) 



12 



Equivalently, to sample {x,ti, . . . ,td) from /x^'^^ first take a sample {x,ti, . . . ,td-i) from 
^(d-i)^ then take a sample (y, t'^, . . . , from fj,^'^^^^ conditioned on t'- = ti for each 

i G [d — 1], and finally set td = y — x (our sample is then {x,ti, . . . ,td-i,tii)). Notice 
that the distribution of a sample {x,ti, . . . , td) from /i(^) is such that for each S Q [d], the 
distribution of the point x + X^jg^ is precisely //. 

For a function f : H C and t € H"^, we define its d^^ -derivative in directions t to be the 
function D^f ■ H ^ C given by 

where equals the complex conjugate a if IS"! is odd, and equals a otherwise. From the 
definition it immediately follows that D(^^ y^f{x) = Dtf{x)Dtf{x + u) (where (t,-u) denotes 
the vector (ti, . . . , td, u) G 

We now define the //-Gowers norm. 



Definition 4.2 (/i-Gowers Norm) If ^ is a distribution on H , and f : H 
its {d, fj,)-Gowers norm by 



I, we define 



E 



{x,t)^f,W [{Dtf){x)] 



When H is of the form Z™, then the (d, ^)-Gowers norm of a function is supposed to estimate 
the correlation, under fi, of that function with polynomials of degree d — 1. Intuitively, this 
happens because the Cowers norm of / measures how often the d^^ derivative of / vanishes. 

The next few lemmas enumerate some of the useful properties that /i-Gowers norms enjoy. 



Lemma 4.3 Let f : H 



Then, 

K^f. [fix) 



< 
< 



[/d+i^^. The lemma follows by noting 



Proof We prove that for every d, ||/||c/<2^ 
that ||/||t;o,^= |E,^^[/(x)][. 

The proof proceeds (following Gowers [GowOl] and Green- Tao |GT08] ) via the Cauchy 
Schwarz inequality. 



2d+l 



E 



[Dtfix)] 



|Ex [Dtf{x)]\ 



Dtf{x)Dtf{y) 



E 



x,t,u 



Dtf{x)Dtf{x + u) 



^E(x,t,«)~Aj{<*+i) 



E 



(a;,t,u)~/i{'i+i) 



Dtf{x)Dtf{x + u) 
[Dit,u)f{x)] 



by Cauchy-Schwarz 

where y is an independent sample of x given t . 
where u = y — x 

by definition of fj,^'^^^^ 
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This proves the lemma. □ 
Definition 4.4 For each i e[r], let gi : H ^C. We define i<S>i=i gi) ■ H'' ^ C by 

I (^Si I {xi,...,Xr) = Ylgi{Xi). 
\i=l ) i=\ 

For each i G [r], let jii he a probability measure on H. We define the probability measure 
01=1 W on H"" by 

I (^/ii I {xi,...,Xr) = Yifiiixi). 

\i=l J i=l 

Lemma 4.5 || giWu^ „i = IlLi hiWu^m- 

Proof Follows by expanding both sides and using the fact that (0i=i Mi)*^'^'' = ®\=i ('^1°'^) 

□ 



Lemma 4.6 Let q > 1 be an integer and let lo E C he a primitive q^^-root of unity. For 
all / : — > C, all probability measures /x on Z^, and all polynomials h G 1'q[Yi, . . . , ¥„] of 
degree < d, 

The above lemma follows from the fact that (Dtf) = (-Dt(/ • ^'^))- 

Lemma 4.7 Let a e Zq \ {0} and let g -."Lq^ C be given by g{y) = ti;"nf=i!/i. Let jj, he a 
probability distribution on Z^ with supp(/i) D {0,1}'^. Then \\g\\ud^n < 1 — e, where e > 
depends only on q,d and fj,. 

Proof As {0,1} C supp(/Li), the distribution n^'^^ give some positive probability 5 > 
to the point (xo,e) = {xo,ei, . . . ,6^), where xq = G Z^, and G Z^ is the vector with 

1 in the ith coordinate and in all other coordinates (and S depends only on q, d and 
/i). Then {Deg){xQ) = n^cfd] fi'(SieS' ^*)°^ ~ '^^^ ^ (since whenever S 7^ [d], wc have 
gillies ^i) = 1). On the other hand, whenever t G (Z^)'^ has some coordinate equal to 
0, which also happens with positive probability depending only on d, fi and we have 
{Dtg{x)) = 1. Thus in the expression 



\\g\\ud,_ 



J_ 

2d 



since every term in the expectation has absolute value at most 1, and wc just found two 
terms with positive probability with values 1 and u;^" 7^ 1, we conclude that H^Hf/d^^ < 1 — e 
for some e depending only on g, /x and d. □ 

We now put together the above ingredients. 
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Theorem 4.8 Let f : {Z'^f 



C be given by 



f{xi,...,Xr) =UJ^i 

where aj G \ {0} for all j € [r]. Let fi be a probability distribution on with supp(^) ^ 
{0, l}*^. Then for all polynomials h G '^q[{Yij)ie[d],je[r]]! deg(/i) < d, we have 



E, 



f{x)uj 



h(x) 



where c < 1 depends only on q,d and ji. 

Proof Let gj : Zg ^ C be given by gj{y) = w'^j 11^=1 w ^a,s in in Lemma l4.7p . and take 
c = 1 — e from that Lemma. Notice that / = ®'j^igj. Therefore by Lemma l4.5t we have 

r 

ii/iic/d.M®'- = n hjWud,^, < c". 

As the degree of h is at most d — 1, Lemma 14.61 imphes that 



Ifu^' 



Lemma 14.31 now imphes that 



E, 



f{x)uj 



h(x) 



as desired. 



□ 



We can now complete the proof of Lemma 14.11 

Proof of Lemma I4.lt By fixing the variables Zi for i ^jEj, and then averaging 
over all such fixings, it suffices to consider the case [m] = DjEj. Then the polynomial 

Q{Zi, . . . , Zm) = (jyj=i Hje^Bj + ^{^) can be rewritten in the form (after renaming 
the variables): 

r d 

where deg{h) < d. Let fi be the p-biased probability measure on {0, l}*^ C Z^. Theorem HSl 
now implies that 



'^Xr^H'^^ 



Q(x) 



< 2^^9,P,d('^) 



as desired. 



□ 
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5 Proof of Theorem 12.3 



The proof of Theorem 12.31 wih be via a more general theorem amenable to inductive proof, 
Theorem 15.81 Just as Theorem 12.31 states that for almost all G € G{n,p), the truth of any 
FO[Modq] sentence on G is determined by subgraph frequencies, freq^. Theorem 15.81 states 
that for almost all graphs G G G{n,p), for any wi, . . . ,Wk € Vq the truth of any FO[Modg] 
formula 92(^1, . . . , Wk) on G is determined by the adjacency and equality information about 
wi, . . . ,Wk (which we will call the type), and the labelled subgraph frequencies at w. In the 
next subsection, we formalize these notions. 

5.1 Labelled graphs and labelled subgraph frequencies 

Let I be a finite set. We begin with some preliminaries on /-labelled graphs. 

Definition 5.1 (/-labelled graphs) An /-labelled graph is a graph F = {Vf,Ef) where 
some vertices are labelled by elements of I, such that (a) for each i £ I, there is exactly 
one vertex labelled i. We denote this vertex F{i), and (b) the graph induced on the set of 
labelled vertices is an independent set. We denote the set of labelled vertices of F by C{F). 

Definition 5.2 (Homomorphisms and Copies) A homomorphism from an I-labelled 
graph F to a pair (G,w), where G is a graph and w G Vq, is a homomorphism x ^ 
h\om{F,G) such that for each i £ I, x mo-ps F[i) to Wi. A homomorphism from F to 
{G,'w) is called injective if for any distinct v,w (z Vp, such that {v,w} % "we have 

xiv) 7^ x(^)- ^ copy of F in (G, w) is a set E C Eq such that there exists an injective 
homomorphism x from F to (G,w) such that E = x{Ef) ■= {{xi'^) ^ xi'w)) \ iv,w) G Ep}. 
An automorphism of F is an injective homomorphism from F to {F,w), where Wi = F{i) 
for each i G /. 

Definition 5.3 (Hom, Inj, Cop, Aut for labelled graphs) Let F be an I -labelled graph, and 
G be any graph. Let w G Vq. We define Hom(/^, (G, w)) to be the set of homomorphisms 
from F to (G, w). We define lnj(/^, (G, w)) to be the set of injective homomorphisms from 
F to (G,wr). We define Cop(F, (G, w)) to be the set of copies of F in (G,w). We define 
/Kut{F) to be the set of automorphisms of F. We let [F](G, w) (respectively (-F)(G, w), 
aut{F)) be the cardinality 0/ lnj(F, (G, w)) (respectively Cop(F, (G, w)), Aut{F)). 

Finally, let [F]g(G,w) = [F](G,w) mod q and (F)g(G,w) = (F)(G,w) mod q. 

Definition 5.4 (Label-connected) For F an L -labelled graph, we say F zs label-connected 
if F\£{F) is connected. Define Conrij to be the set of all L -labelled label- connected graphs 
with at most t unlabelled vertices. For i £ I, we say an L-labelled graph F is dependent 
on label i if F[i) is not an isolated vertex. 

Definition 5.5 (Partitions) If I is a set, an /-partition is a set of subsets of I that are 
pairwise disjoint, and whose union is I. IfH is an I partition, then for i £ I we denote the 
unique element ofH containing i by Il{i). IfV is any set and w G , we say w respects 
n if for all i,i' £ I, Wi = Wi' iffll{i) = n(i'). 
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The collection of all partitions of I is denoted Partitions(/). 

If / C J, n € Partitions(/) and 11' G Partitions( J), we say 11' extends 11 if for all ii,i2 € /, 
n(zi) = n(i2) if and only if = Ii'{i2). 



Definition 5.6 (Types) An I-type t is a pair (JlT-,Er) where lir G Partitions(I) and 
Et ^ (^^)- For a graph G and w G Vq, we define the type o/ w in G, denoted typefj(w), 
to he the /-type t, where w respects lir, and for all i,i' G I, {n^(i), n,-(i')} G Er if and 
only if Wi and Wii are adjacent in G. 



The collection of all /-types is denoted Types(/). 

If / C J, and r G Types(/) and r' G Types(J), we say r' extends r if 11,-/ extends 11^ and 
for each ii,i2 G /, {n^(ii), 11^(^2)} G E-j- if and only if {11^/ (ii), 11^/(^2)} G E^-i. 



Definition 5.7 (Labelled subgraph frequency vector) Let G he a graph and I he any 
set. Let w G Vj|. We define the labelled subgraph frequency vector at w, freq^(w) G Zg°""^, 
to he the vector such that for each F G Conrij, 

(freq^(w))^ = [F],(G,w). 



Remark We will often deal with [/cj-labelled graphs. By abuse of notation we will refer 
to them as A:-labelled graphs. If w G V^^'^ and n G when we refer to the tuple (w, v), we 
mean the [A;-|-l]-tuple whose first k coordinates are given by w and whose A: + lst coordinate 
is V. Abusing notation even further, when we deal with a [A: + l]-labelled graph F, then by 
[F](G,w,u), we mean [F](G, (w, u)). Similarly Conn^ denotes Conn|^j. 



5.2 The quantifier eliminating theorem 



We now state Theorem 15.81 from which Theorem 12.31 follows easily. Informally, it says that 
an FO[Modg]-formula (/^(w) is essentially determined by the type of w, typeG(w), and the 
labelled subgraph frequencies at w, freq^(w). 



Theorem 5.8 For all primes q and integers k,t > 0, there is a constant c = c{k, t, q) such 
that for every FO[Modq] formula (p{ai, . . . ,ak) with quantifier depth t, there is a function 

if) : Types(A;) x Zq°""'' — > {0, 1} such that for all p G (0, 1), the quantity 



Pr 

GeG{n,p) 



ywi, ...,Wke Vg, 

{G satisfies (p{wi, ... 



Wk)) ^ (^(typeG(w),freq^(w)) = 1) 



> 1 



-n(n) 



Putting k = 0, we recover Theorem 12.31 

We now give a brief sketch of the proof of Theorem 15.81 (the detailed proof appears in 
Section [6]). The proof is by induction on the size of the formula (p. When the formula f 
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has no quantifiers, then the truth of (/^(w) on G is completely determined by typec(w). 
The case where if is of the form Lpi{ai, . . . ,ak) A 932(011 • • • 1 Ofc) is easily handled via the 
induction hypothesis. The case where </?(ai, . . . , au) = -^(pi{ai, . . . , Ok) is similar. 

The key cases for us to handle are thus (i) . . . , Ofc) is of the form Modg/?, f'{ai, . . . , ak, f3), 
and (ii) (p{ai, . . . , a^) is of the form 3/3, (p'{ai, . . . , a^, We now give a sketch of how these 
cases may be handled. 

For case (i), let Tp' : Types(/c+ 1) x '''^^ be the function given by the induction hypothesis 
for the formula (p' . Thus for most graphs G G G{n,p) (namely the ones for which ip' is 
good for (/?'), ip{wi, . . . ,Wk) is true if and only the number of vertices v G Vq such that 
■i/''(type(^(w, w), freqQ(w, w)) = 1 is congruent to i mod q. In Theorem 16.11 (whose proof 
appears in Section [7j), we show that the number of such vertices v can be determined solely 
as a function of typeg(w) and freq^(w) for suitable a. This fact allows us to define ^ in a 
natural way, and this completes case (i). 

Case (ii) is the most technically involved case. As before, we get a function ip' corresponding 
to if' by the induction hypothesis. We show that one can define ip essentially as follows: 

define ipir, /) = 1 if there exists some (r', /') G Types(A; + 1) x Zg that "extends" (r, /) 

for which iP'{t' , /') = 1; otherwise ip{T, f) = 0. Informally, we show that if it is conceivable 
that there is a vertex v such that 93' (w, v) is true, then (p{w) is almost surely true. Proving 
this statement requires us to get a characterization of the distribution of labelled subgraph 
frequencies, significantly generalizing Theorem 12. 4[ This is done in Theorem 16.121 (whose 
proof appears in Section [8]). 

6 Quantifier Elimination 

In this section, we give a full proof of Theorem 15. 8[ Before doing so, we state the main 
technical theorems: Theorem 16.11 (which is needed for eliminating Modg quantifiers), and 
Theorem 16.121 (which is needed for eliminating 3 quantifiers) . We do this in the following 
two subsections. 

6.1 Counting extensions 

The next theorem plays a crucial role in the elimination of the Modg quantifiers. This is the 
only step where the assumption that (7 is a prime plays a role in the modular convergence 
law. 

Theorem 6.1 Let q be a prime, let k,b > be integers and let a > {q — 1) ■ b ■ [Conn^_,_]^|. 
There is a function 

X : Types(A: + 1) x x Types(A:) x ^ 

such that for all t' G Types(A; + 1), /' G r G Types(A;), / G Zg°""*, it holds that 

for every graph G, and every wi, . . . ,Wk G Vq with type(j(w) = r and freqQ(w) = /, the 
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cardinality of the set 

{v gVg: typeclw, v) = t' A freq^(w, v) = /'} 
is congruent to A(r', /', r, /) mod q. 

The proof appears in Section [71 The principal ingredient in its proof is the following lemma, 
which states that the numbers [F](G,w), as F varies over small label-connected graphs, 
determine the number [F']{G,w) for all small graphs F' . 

Lemma 6.2 (Label-connected subgraph frequencies determine all subgraph fre- 
quencies) For every k-labelled graph F' with \Vp' \ £.{F')\ < t, there is a polynomial 
6p' G ^[(-^^Fj^GConn* ] such that for all graphs G and w G Vq, 

[F']{G,^)=5f'{x), 
where x € is given by xp = [F]{G,w). 

6.2 The distribution of labelled subgraph frequencies mod q 

In this subsection, we state the theorem that will help us eliminate 3 quantifiers. Let us first 
give an informal description of the theorem. We are given a tuple w G [n]^, and distinct 
ui, . . . ,Us G [n] \ {wi, . . . ,Wk}- Let G be sampled from G{n,p) (recall that we think of 
G{n,p) as a random graph whose vertex set is [n]: thus the Wi and Uj are vertices of G). 
The theorem completely describes the joint distribution of the labelled subgraph frequency 
vectors at all the tuples w, (vi^, ui), . . . , (w, u^); namely it pins down the distribution of 
(freq^(vif), freq^(w, til), ■ • • > f''eqf;(vif, n^)). We first give a suitable definition of the set of 
feasible frequency vectors, and then claim that (a) the freq^(w) is essentially uniformly 
distributed over the set of its feasible frequency vectors, and (b) conditioned on freq^(w), the 
distributions of freqg-(w, ui), . . . , freqf;(w, Ug) are all essentially independent and uniformly 
distributed over the set of those feasible frequency vectors that are "consistent" with freq^.. 

To define the set of feasible frequency vectors (which will equal the set of all possible values 
that freqg(w) may assume), there are two factors that come into play. The first factor, 
one that we already encountered while dealing with unlabelled graphs, is a divisibility 
constraint: the number [F]{G,w) is always divisible by a certain integer depending on F, 
and hence for some F, it cannot assume arbitrary values mod q. The second factor is a 
bit subtler: when wi, . . . ,Wk are not all distinct, for certain pairs F,F' of label-connected 
fe-labelled graphs, [F](G, w) is forced to equal [F'](G, w). Let us see a simple example of 
such a phenomenon. Let k = 2 and let wi = W2- Let the 2-labelled graph F be a path of 
length 2 with ends labelled 1 and 2. Let the 2-labelled graph F' be the disjoint union of an 
edge, one of whose ends is labelled 1, and an isolated vertex labelled 2. Then in any graph 
G, [F](G, w) = [F'](G, w) = the degree of wi. 

In the rest of this subsection, we will build up some notation and results leading up to a 
definition of feasible frequency vectors and the statement of the main technical theorem 
describing the distribution of labelled subgraph frequency vectors. 
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Definition 6.3 (Quotient of a labelled graph by a paritition) Let F be a I-lahelled 
graph and let IT G Partitions(7). We define F/Il to be the Il-labelled graph obtained from F by 

(a) for each J G IT, identifying all the vertices with labels in J and labelling this new vertex 
J, and (b) deleting duplicate edges. If F and F' are I-labelled graphs andii G Partitions(/), 
we say F and F' are Il-equivalent if F/U. = F' /II. 

Let w eVQ. Let IT G Partitions(/) be such that w respects 11. Define (w/II) G by: for 

each J G n, (w/n)j = wj, where j is any element of J (this definition is independent of 
the choice of j G J). Observe that as J varies over H, the vertices {w/H)j are all distinct. 

The next two lemmas show that the numbers [F] (G, w) must satisfy certain constraints. 
These constraints will eventually motivate our definition of feasible frequency vectors. 

Lemma 6.4 If G is a graph andw G Vq, with w respecting H G Partitions(J), then for any 
I-labelled F, 

[F](G,w) = [F/n](G,(w/n)). (2) 

Proof We define a bijection a : \n}{F/U, (G, w/H)) ^ lnj(F, (G, w)). Let tt G Hom(F, F/U) 
be the natural homomorphism sending each unlabelled vertex in Vp to its corresponding 
vertex in Vp/n, and, for each i G / sending F{i) to (F/n)(n(i)). We define a{x) to be x°'^- 

Take distinct x,x' G lnj(^/n, (G, w/II)). Let u G Vp/n with xiu) ^ x'iu). Note that 
u cannot be an element of >C(F/n), for if u = (F/n)(n(z)), then x(^) = x'(^) = '^^i- 
Thus u ^ C{F/Il). Let v E Vp he the vertex tt~^{u) (which is uniquely specified since 
u ^ C{F/U)). Thus we have x(7r(?;)) = x(n) 7^ x'{u) = x'(7r('y)). Thus a{x) + a(x')> and 
a is one-to-one. 

To show that a is onto, take any x S lnj(F, (G, w)). Define x' S lnj(F/n, (G, w/II)) by: 

1. x:{u) = x{j^-^{u)) ifn0£(F/n). 

2. x'(u) = for any j G J, if u = (F/n)(J) with J G H. 

Then a(x') = X- □ 



Lemma 6.5 Let G be a graph and w G V^^. Suppose all the {wi)iQj are distinct. Let F be 
an I-labelled label- connected graph with \Ef\ > 1. Then 

[F](G,w)=aut(F)-(F)(G,w). 

Proof Wc give a bijection a : Aut(F) x Cop(F, (G, w)) lnj(F, (G, w)). 

For each E G Cop(F, (G, w)), we fix a xs G lnj(-^, (G, w)) such that xe{Ef) = E. Then we 
define a((T, E) = xe ° cr- 
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First notice that a{a,E){Ep) = xe{(^{Ef)) = Xe{Ef) = E. Thus if a{a^E) = a{a',E'), 
then E = E'. But since xe is injective, for any a 7^ a' , we have xe o cj 7^ xe ° o"'. Thus a 
is one-to-one. 

To show that a is onto, take any x ^ (G, w)). Let E = xiEp)- As F is label- 

connected and xe{Ef) = x{Ef), we have XEiVp) = x(^f)- We may now define a E Aut(F) 
by a{u) = XE^ixiu)) for each u € Vp. Clearly, a{a, E) = x, and so a is onto. 

Thus a is a bijection, and the lemma follows. □ 

Note that Lemma 12.21 and Lemma |3. II follow formally from the above lemma. 

Let Ki(/) be the /-labelled graph with |/| + 1 vertices: |/| labelled vertices and one isolated 
unlabelled vertex. The role of Ki(/) in the /-labelled theory is similar to the role of Ki in 
the unlabelled case. 

Definition 6.6 (Feasible frequency vectors) We define the set of feasible frequency 
vectors, FFreq(T, /, a) to be the set of f & such that 

(a) for any F € Conn", we have fp € aut(F/nT-) • Z^. 

(h) for any F,F' G Connf that are 11^- equivalent, we have fp = fp'- 

Let FFreq^(r, /, o) be the set {/ G FFreq(r, /, o) : fKi{i) = n — |n^| mod q}. Note that if 
n = n' mod q, then FFreq„(T, /, a) = FFreq„/(r, /, a). 

Observe that for any w G Vq with type(j(w) = r, the vector freqQ(w) is an element of 
FFreq(r, /, o). This follows from Lemma 16.41 and Lemma 16.51 which allow us to deduce 
(recall that (w/n^)j are all distinct for J G 11,-) that for any F G Conn J, 



Observe also that if \Vg\ = n, then freqc.(w) G FFreq„(T, /, a), since [Ki{I)]{G,w) = 
\Vg \ {wi, . . . ,Wk}\ = n — |ntype(^-) [, as required by the definition. 

Definition 6.7 (Extending) Let L be a set and let J = LU{i*}. Let a >b > be positive 
integers. We say (r', /') G Types( J) x FFreq(r', J, b) extends (r, /) G Types(/) x FFreq(T, /, a) 
if t' extends r, and for every F G Conn^, we have 



where F is the graph obtained from F by introducing an isolated vertex labelled i* . 



[F](G,w) = aut(F/no • (F/n,)(G,w/no. 



(3) 



1. lf{i*}^Iir', 



(4) 



2. if{i*} G n^', letting 6h ■ 



Conn^^ 



be the function given by Lemma \6.S\ 




(5) 



u&Vf\C(F) 



where 
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• F is the graph obtained from F by introducing an isolated vertex labelled i* . 

• Cu equals 1 if for all i £ I, if u is adjacent to F{i), then {11^' {i*) ,^t' {i)} & Et' ■ 
Otherwise, Cu = 0. 

• Fu is the graph obtained from F by labelling the vertex u by i* and deleting all 
edges between u and the other labelled vertices of F . 

The crux of the above definition is captured in the following lemma. 

Lemma 6.8 Let G be a graph. Let a > b > be integers. Let w € and v £ V. Let 
T = typec.(w), t' = typeg(w, f), / = freq^(w) and f = freqQ(w,f). Then {t' , f) extends 
(rj). 

Proof We keep the notation of the previous definition. First observe that r' extends r. 

If {A; + 1} ^ n^/, then we need to show that [F]q{G, w) = [F]q{G, w, v) for each F G Conn^. 
This is immediate from the definitions. 

U{k+1} G n^/ , then we need to show that [F]q{G,w) = [F]q{G,w,v)+Y^^^y^'^^p^Cu[Fu]qiG,w,v) 
We do this by counting the x ^ (G,w)) based on its image x{^f) as follows: 

1. Category 1: v ^ xi^p)- There are precisely [F]{G,w,v) such x- 

2. Category 2: v = xi^) (in this case u is uniquely specified). Note that u ^ J-'iF). 
Then it must be the case that for any i € [k] such that u is adjacent to F{i), Wi is 
adjacent to v. Thus {IlT-'{i),Ilr'{k + 1)} G E^-/, and so = 1. The number of such x 
is [Fu]{G,w,v). 

This proves the desired relation. □ 

We now state and prove two key uniqueness properties enjoyed by the notion of extension. 

Lemma 6.9 Let a > b > be integers. Let w G Vq. Let u £ Vg \ {wi, . . . ,vuk}. Let 
T = type(^(w) and t' = type(^(w, u). Let f = freq^(w). Then freqfj(w,u) is the unique 

f G Zq°""'^+' such that: 

• for each H G Conn^_,_]^ that is dependent on label k + 1, we have f'^ = [H]q{G,w,u). 

• (t',/') extends {T,f). 

Proof By Lemma 16.81 the vector freqQ(w,n) is such an /'. 

To prove uniqueness, it suffices to show that any /' satisying these two properties equals 
freqQ(w,n). Thus it suffices to show that for any H G Conn^^^ not dependent on label 
k + 1, f'H = iheq'G{^,u))H. 
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We prove this by induction on \Vh \ C{H)\. Let H € Conn^^^ not dependent on label 
A; + 1. Thus H is of the form F for some graph F S Conn^ (as in the previous lemma, 
for a [fc]-labelled graph F, we let F be the [k + l]-labelled graph obtained by adjoining an 
isolated vertex labelled k + 1 to F). By Equation ([5]), we see that f'jj is uniquely determined 
by r, r', fp and the numbers (f'jji) „, ^ \Vh\c(h)\-i (since each Cu is determined by r' and 

each of the graphs Fu have \Fu \ C{Fu)\ < \Vh \ ^{H)\ — 1). By induction hypothesis, 
all the f'jf, = (freqfj(w, Thus, since freqg.(w, also satisfies Equation ([5]), we have 

fii ~ (^^''^^g("^' ' required. □ 



Lemma 6.10 Let a > b > be integers. Let {t, f) G Types(A;) x FFreq(T, [A;],a). Let r' € 
Types(A; + l) extend t with {A: + l} H^/. Then there is at most one f G FFreq(T', + 
such that (r', /') extends (t, /). 

Proof As in the previous lemma, for a [A;]-labelled graph F, we let F be the [/c+l]-labelled 
graph obtained by adjoining an isolated vertex labelled A: + 1 to F. For any F G Conn^, 
we must have f'~ = fp. Now we claim that any H G Conn^ is H-equivalent to some graph 

of the form F. To prove this, let j G [k] be such that n^'(j) = U^-'ik + 1). Let H* be 
the graph obtained from H by adding, for each neighbor u of H{k + 1), an edge between u 
and the H{j), and then removing (a) all edges incident on H{k + 1), and (b) any duplicate 
edges introduced. By construction, H/Ht-i = H* /Ht-i, and so = by Equation ([3]). 
In addition, the H*{k + 1) is isolated, and hence H* is of the form F for some F G Conn^. 

What we have shown is that for every H G Conn^_,_]^, f'^j is forced to equal fp for some 
F G Conn|. This implies that /' is specified uniquely. □ 

Finally, we will need to deal with random graphs G{n,p) with some of the edges already 
exposed. The next definition captures this object. 

Definition 6.11 (Conditioned Random Graph) Let A = {Va,Ea) be a graph with 
Va Q [n]. We define the conditioned random graph G{n,p \ V^,£'^) to be the graph 
G = {Vg,Eg) with Vg = [n] and Eq = Ea^JE', where each {i,j} G ('2^) \ {^^) is included 
in E' independently with probability p. 

We can now state the main technical theorem that describes the distribution of labelled 
subgraph frequencies, and will eventually be useful for eliminating 3 quantifiers. 

Theorem 6.12 Let a > b be positive integers. Let A be a graph with Va Q [n] and \Va\ < 
n' < n/2. Let G G G{n,p \ Va, Fa). Let w = {wi, . . . , w^) G V^, and let ui, . . . ,Us G 
Va \ {wi, . . . ,Wk} be distinct. Let r = type(j(w) and let Ti = typeQ(w,nj) (note that 
T,Ti, . . . ,Ts are already determined by Ea)- Let f denote the random variable freq^(w). 
Let fi denote the random variable freq^(w, Uj). 

Then, there exists a constant p = p{a,q,p) > 0, such that if s < p ■ n, then the distribu- 
tion 0/ (/, /i, . . . , fs) over FFreq„(T, [k],a) x fl- FFreq„(Tj, [k + 1], 6) is 2'^^"'^ -close to the 
distribution of {h, hi, ... , kg) generated as follows: 
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1. h is picked uniformly at random from FFreq„(r, [A;], a). 

2. For each i, each hi is picked independently and uniformly from the set of all f € 
FFreq„(ri, [A: + 1], h) such that (r,, /') extends (r, h). 

6.3 Proof of Theorem EH 

We now prove Theorem 15 .8^ where the main quantifier ehmination step is carried out. 

Theorem 15.81 (restated) For every prime q and integers k,t > 0, there is a constant 
c = c{k,t,q) such that for every FO[Moclg] formula ip{ai, . . . ,ak) with quantifier depth t, 
there is a function •i/' : Types(fc) x {0, 1} such that for all p G (0, 1), the quantity 



Pr 

GeG(n,p) 



{G satisfies ip{uJi, . . .,Wk)) (V'(typeG(w), freqG'(w)) = 1) 



> 1 — 2^^("). 



Proof The proof is by induction on the size of the formula. If ^p{wi, . . . , Wk) is an atomic 
formula, then trivially there exists a ^jj : Types(A;) — > {0, 1} such that for every graph G and 
every w G Vq, the statement '^{wi, . . . , Wk) holds if and only if ^(typegr(w)) = 1. Thus we 
may take c{k, 0, q) = 0. We will show that one may take c{k, t, q) = (q — l)c{k + l,t — l,q) ■ 

2c{k+l,t-l,qf 

Now assume the result holds for all formulae smaller than (p. 

Case A: Suppose 9?(ai, . . . , a^) = ipi{ai, . . . , ak)A(p2{ai, . . . , a^). By induction hypothesis, 
we have functions ^i, "02 and a constant c such that PrG'[Vi(;i, . . . ,Wk & Vq, {ifi{wi, . . . , Wk) ^ 
V'i(typeG(w),freq^(w)) = 1)] > 1 - 2-^(") and Pr^ [Vu-i , . . . , u;^ G Vg, (Mwi, ■ ■ ■ ,Wk) ^ 
V'2(typeG(w),freq^G(w)) = 1)] > 1 - 2-^("). Setting V'(r,/) = Mr, f) ■ Mr J), it follows 
from the union bound that 

Pr[Vt/;i, ...,WkeVG, {ifiwi,. ..,Wk)^ V'(typeG(w), freq^(w)) = 1)] > 1 - 2-^("). 

(jr 

Case -i: Suppose ip{ai, . . . ,ak) = ^ip'{ai, . . . ,ak). Let i/j' : Types(A;) x Z^"""*" {0,1} 

be such that Pre [Vwi , . . . , -Wfc G Vg,{(p'{wi, . . . ,Wk) V'(typeG'(w), freqG(w)) = 1)] > 
1 _ 2-f^{"). Setting V(t, /) = 1 - i^'ir, /), we see that 

Pi[ywu . . . G Vg, {^{wu. ..,Wk)^ V'(typeG(w),freq^(w)) = !)]>!- 2-^("). 

(j 

Case Modgi Suppose (p{aij . . . , ak) = Modg/3, (p\ai, . . . , ak^P). Let c' = c{k + 1, t — 1, 
and let i/j' : Types(A: + I) x Zg {0, 1} be given by the induction hypothesis, so that 

FT[\fwi,...,Wk,v G VG,i^'iwi,...,Wk,v) ^^lj'{typeaiw,v),heq''Giw,v)) = 1)] > l-2-^("). 

Call G good if this event occurs, i.e., if 

\/wi,...,Wk,v G VG,{(p'{wi,...,Wk,v) 4^ ip'{typeG{w,v),hec\Q{w,v)) = 1). 
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Let 7(1^1, . . . , Wk) be the number (mod q) of v such that ip'iwi, . . . , Wk, v) is true. Then for 
any good G (doing arithmetic mod q), 

'y{wi,... ,Wk) = ^'(typeG.(w,v),freq^(w,u)). 
Grouping terms, we have 

j{wi,...,Wk) = Y ^'('^''/O • G : typeG(w,?;) = r') AfreqG(w,u) 

r'eTypes{A:+l) Conn?' , 

= Y /') • A(t', typeG(w), freq^(w)) 

r'J' 

(applying Theorem 16. 11 and taking c = {q — l)c'2('^')^) 

which is solely a function of typeQ(w) and freq^(w). Thus, there is a function ^ : Types(/c) x 
1^^°""k _j. such that for all good G and for all wi, . . . ,Wk € Vq, V'(typec(w), freqQ(w)) = 

1 if and only if 7(w) = i mod q. Thus, 

Pr[Vt(;i,...,ti;fc,((Mod^t;,(/p'(tyi,...,ti;fc,t;)) 4^ V(typeG(w),freq^(w)) = 1)] > 1 - 2 
as desired. 



-n{n) 



Case 3: Suppose (p{ai, . . . , a^) = 3/3, (p'{ai, . . . , Ok, P)- Let c' = c{k + l,t — l,q) and let 
ip' : Types(/i: + 1) x ^ {0, 1} be such that 



Pic[ywi,...,Wk,v G VG,iy:>'{wi,...,Wk,v) 44> V''(typeG(w, u), freqc(w, v)) = 1)] > 1-2 



-n(n) 

(6) 



For this case, we may choose c to be any integer at least c'. Define tp : Types(A;) x Zg"""*" — > 
{0, 1} by the rule: ipir, f) = 1 if there is a (r',/') G Types(A; + 1) x FFreq„(r', [k + l],c') 
extending (r, /) such that ip'{T',f') = 1. 

Fix any w G [n]'^. We will show that 
Pr[(37;,^'(typeG(w,7;),freq^(w,z;)) = 1) ^ ^/J{typeaMM^GM) = 1] > 1 - 2-^^"). (7) 

Cr 

Taking a union bound of ([7]) over all w G [n]^, and using Equation ([6]), we conclude that 
Pr Nwi,...,Wk G VG,{'P{wi,...,Wk)^i^{typeGM,^reqGM) = 1)] > l-2-^("), 

GeG{n,p) 

as desired. 

It remains to show Equation ([7]). It will help to expose the edges of the random graph G 
in three stages. 

In the first stage, we expose all the edges between the vertices in {wi, . . . , Wk}- 
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For the second stage, let s = p{c, q,p) ■ n (where p comes from Theorem I6.12p and pick 
distinct vertices ui, . . . ,Us G [n]\ {wi, . . . ,Wk}- In the second stage, we expose all the unex- 
posed edges between the vertices in {wi, . . . ,Wk,ui, . . . ,Us} (i.e., the edges between UjS and 
WjS, as well as the edges between the UiS and Ujs). Denote the resulting graph induced on 
{wi, . . . , Wk,ui, . . . , Us} after the second stage by A (so that Va = {wi, . . . Wk, ui, . . . , Us}). 

In the third stage, we expose the rest of the edges in G. Thus G is sampled from G{n,p \ 
Va,Ea). 

Let r denote the random variable type(;(w). Note that r is determined after the first 
stage. Let ri, . . . , denote the random variables type(j(w, ui), . . . , typeQ{w, Us). Note that 
Ti,...,Ts are all determined after the second stage. Let / denote the random variable 
freqQ{w). Let /i, . . . , /<j denote the random variables heqQ{w, ui), . . . , freqc.(w, Us). The 
variables /, /i, . . . ,/<j are all determined after the third stage. Notice that the content of 
Theorem 16. 121 is precisely a description of the distribution of (/, /i, . . . , J^). 

We identify two bad events Bi and i?2. 

Bi is defined to be the event: there exists a € Types(fc + 1) extending r, with {A; + 1} G 11^ 
(ie, types a where vertex k + 1 is distinct from the other vertices), such that 

\{i G [s] : Ti = a}\ < min{p^ (1 - p)''}. 

(This can be interpreted as saying that the type a appears abnormally infrequently amongst 
the Ti). Note that for any a extending r, the events 'Vj = a", for i G [s], are independent 
conditioned on the outcome of the first stage, since they depend on disjoint sets of edges of 
G. Also, for each i and each a extending r with {fc + 1} G Ha, the probability that Ti = a 
is > m.m{p^, (1 — p)''}. Therefore, applying the Chernoff bound, and taking a union bound 
over all a extending r with {/c + 1} G H^, we see that 

Pr[Bi] < 2^exp(-smin{/,(l -p)'^}) < 2-^("). 

Now let 

S = {{a,g) e Types(A: + 1) x FFreq„ (cr, [k + l],c') | {/c + 1} G 
AND {a,g) extends (r, /) AND i;'ia,g) = 1}. 

B2 is defined to be the event: S ^% and for each i G [s], (tj, fi) S. We study the probabil- 
ity of -'B1AB2. Let U be the set of {d,di, . . . ,ds) G FFreq„(r, [A;], c) xHi FFreq„(rj, [/c-|-l], c') 
such that 

1. The set S{d) defined by 

S{d) = {{c7,g) G Types(A; + 1) x FFreq„(cr, [k + l],c') [ {A; + 1} G 
AND {a,g) extends (r, d) AND 'ip'ia,g) = 1}, 

is nonempty. 

2. For each i G [s], {n.di) ^ S{d). 
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By definition, the event B2 occurs precisely when (/, /i, . . . , fs) G U. 

By Theorem I6.12| for any fixing of Ea, the probability that (/, /i, . . . , /s) G C/ is at most 
2-f2(n) j^iore than the probability that {h,hi, . . . ,hs) S U. As the event Bi is solely a 
function of Ea, we conclude that Pr[^i?i A (/, /i, . . . , fs) G U)] < Pr[-^Bi A{h,hi, . . . ,hs) G 
U] + 2-*^^. 

It remains to bound Pr[-i5i A {h, hi, ... , hg) G [/]. If S{h) 7^ 0, take a {a, g) G ^(/i). In the 
absence of the number of i G [s] with = o" is at least ^sminjp*^, (1 — p)^}. For all 
these i, it must hold that hi g in order for {h,hi, . . . , hg) to lie in U . Therefore, 

^ \ |smin{p'=,(l-p)'=} 

FFreq„(r,A; + l,c')|y 
Notice that this last quantity is of the form 2~^P'i'''-'^^^\ 
Putting everything together, 

Prh^i A B2] < Prh^i A {h, hi,..., h^) G C/] + 2-^(") < 2-^(") + 2"^^") < 2-^("). 

Therefore, with probability at least 1 — 2~^(") , the event B2 does not occur. The next claim 
finishes the proof of Equation dTj) , and with that the proof of Theorem 15.81 

Claim 6.13 If B2 does not occur, then 

(3?;,^'(typeG(w,i;),freq^(w,'(;)) = 1) 44> (V'(typeG(w), freq^(w)) = 1). 

Proof Let r = typec.(w) and / = freqf;(w). 

If 4>{t, f) = 0, then we know that for all (r', /') G Types(A;+l) x FFreq„(T', k+1, c') extending 
(r, /), we have ip^r' , /') = 0. Thus by LemmaElBl for all v G V^, V''(type(3.(w, w), freq^(w, ■;;)) = 
0, as required. 

If ^(r, /) = 1, then we consider two situations. 

• The self-fulfilling situation: If there is a (r', /') G Types(A;+l) x FFreq„(r', k+1, d) 
extending (r, /) with {A; + 1} 11,-/ and ^j'ir' , f) = 1. In this case, take any j G [k] 
with n^'(j) = + 1), and let v = Wj. Thus type(^(w, f) = r'. By Lemma Fb-lOi 
since {t' , f) extends (r, /) with {k + 1} 11^', it follows that freq^(w,w) = /'. 
Therefore, with this choice of v, we have ip'{typeQ(\v, u), freq^(w, v)) = 1, as required. 

• The default situation: In this case, there is a (r', /') G Types(A; + l) x FFreq„(T', k + 
l,c') extending (r, /) with {A; + 1} G 11^' and Tp'{T',f') = 1. This is precisely the 
statement that S ^ %. Therefore, by the absence of the event B2, there must be an 
i G [r] such that {ji, fi) G S. Taking v = ui, we see that '(/;'(typec(w, u), freq^(w, v)) = 
1, as required. 

This completes the proof of the claim. □ □ 



Prh^i A(/i,/ii,...,/i,) G [/] < 
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7 Counting Extensions 



In this section we prove Theorem I6.1i 

7.1 Subgraph frequency arithmetic 

We begin with a definition. A partial matching between two /-labelled graphs Fi,F2 is a 
subset rj C (Vp^ \ C-{Fi)) x {Vf2 \ ^-{^2)) that is one-to-one. For two graphs Fi,F2, let 
PMatch(i<i, F2) be the set of all partial matchings between them. 

Definition 7.1 (Gluing along a partial matching) Let Fi and F2 be two I-lahelled graphs, 
and let rj € PMatch(Fi, F2). Define the gluing of Fi and F2 along r], denoted Fi V^F2, to be 
the graph obtained by first taking the disjoint union of Fi and F2, identifying pairs of ver- 
tices with the same label, and then identifying the vertices in each pair of r] ( and removing 
duplicate edges). We omit the subscript when rj = 9. 

We have the following simple identity. 

Lemma 7.2 For any Flabelled graphs Fi, F2, any graph G and any w € Vq: 

[Fi](G,w).[F2](G,w)= [i^i V,F2](G,w). (8) 

?7ePMatch(Fi,F2) 

Proof We give a bijection 

a: lnj(Fi,(G,w)) X lnj(F2,(G,w)) ^ ]J lnj(Fi F2, (G, w)). 

J7ePMatch(Fi,F2) 

Define a(xi,X2) as follows. Let rj = {(^1,^2) G {Vp^ \ C{Fi)) x {Vp^ \ ^(^^2)) I Xi(^^i) = 
X2(^^2)}- Let ii G lnj(Fi,Fi V,, F2) and L2 € lnj(F2,Fi F2) be the natural inclusions. 
Let X G V,, F2,(G,w)) be the unique homomorphism such that for all v € Vfi, 

X ° i^i{v) = Xi{v), and for all v G Vp^, X ° i^2{v) = X2{v). We define a(xi,X2) := X- 

To see that a is a bijection, we give its inverse p. Let r/ S PMatch(Fi, F2) and x G 
lnj(Fi F2, (G,w)). Let li G lnj(Fi,Fi F2) and L2 G lnj(F2,Fi F2) be the natural 
inclusions. Define /?(x) := (x ° '■i) X ° '-2)- 

Then /? is the inverse of a. □ 
We can now prove Lemma 16.21 

Lemma 16.21 (Label-connected subgraph frequencies determine all subgraph fre- 
quencies, restated) For every k-labelled graph F' with \Vp' \ C{F')\ < t, there is a poly- 
nomial 6f' G 1^[{XF)p^Qor\nl\ "^^^^ ^^^^ graphs G and w G Vq, 

[F']{G,w) = 6p,ix), 
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where x E is given by xp = [F]{G,w). 

Proof By induction on the number of connected components of F' \ C{F'). If F' is 
label-connected , then we take 6pi{X) = Xp'- 

Now suppose F' is label-disconnected. Write F' = Fi y F2 where Fi and F2 are both 
/c-labelled graphs, and Fi \ C{Fi) and F2 \ C{F2) have fewer connected components. 

By equation ([8]), for all G and w, 

[FiVF2](G,w) = [Fi](G,w)-[F2](G,w)- ^ [Fi F2](G, w). 

07^r)ePMatch(Fi,F2) 

Observe that for any ^ 0, each graph Fi F2 has at least one fewer label-connected 
component than FiW F2 = F'. Thus, by induction hypothesis, we may take 

6p> (X) = 6p^ (X) • dp^ (X) - ^ ^Fi v^Fa (X) . 

07^r;GPMatch(Fi,F2) 

This completes the proof of the lemma. □ 



7.2 Proof of Theorem \6A] 

Theorem 16.11 (restated) Let q be a prime, let k,b > be integers and let a > {q — 1) ■ b ■ 
|Conn^^;^|. There is a function 

A : Types(A; + 1) x x Types(A:) x ^ Zg 

such that for all t' € Types(fc + 1), /' G r G Types(A;), / G Zg"""*, it holds that 

for every graph G, and every wi, . . . ,Wk G Vq with typeg(w) = r and freqQ(w) = /, the 
cardinality of the set 

{v£Vg: typeG(w, v) = t' A freq^(w, v) = /'} 
is congruent to X{t' , f ,t, f) mod q. 

Proof We describe the function X{t' , f ,t, f) explicitly. If r' does not extend r, then we 
set A(T',/',r,/) = 0. 

Now assume r' extends r. We take cases on whether A; -|- 1 is a singleton in 11,-' or not. 

Case 1: {/c + 1} G 11^/. In this case, there is an / C [k] such that typeQ{wi, . . . , Wk,v) = t' 
if and only if w {wi, . . . ,Wk} and {v,Wi) G Eg <^ i & I (explicitly, I = {i ^ [k] \ 

Let for each n, ?; G Vq, let Xuv £ {0, 1}, where Xuv = 1 if and only if u is adjacent to v in G. 

Then, using the fact that q is prime, the number (mod q) of v with typeQ{'w,v) = r' and 
freqQ(w,f) = /' can be compactly expressed as (doing arithmetic mod q): 

E n^-^ n (1-^-.) n {i-{[FUG,y.,v)-f'Fy-') 

veVG\{wi,...,w^}i&I je[k]\I FGConn* , 
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Expanding, the expression Ilje/ ^1""^ nje[fc]\/(-'- ~ ^vwj) may be expressed in the form 

T.sc[k] bsUies^vw,- Using LemmaO the expression HpgConn^^^i (l " w, w) - f'pf 

may be expressed in the form J2j Cj[-^j]i}(G,w,t;), where each Fj is a + 1-labelled graph 
with at most |Conn^_(_]^| ■ b ■ {q — 1) < a vertices. 

Thus we may rewrite the expression for A(r', /', r, /) as: 

v€[n]\{w,,...,Wk} \ S ieS / \ j J 
S,j ve[n]\{wi,...,w^} \ ViGS / / 

= ^6sc,[Fy,(G,w), 

s,j 

where Fg j is the fc- labelled graph obtained from Fj by 

(a) For each z G S*, adding an edge between the vertex labelled k + 1 and the vertex 
labelled i, and 

(b) Removing the label from the vertex labelled A; + 1. 

Finally, note that by Lemma \U72[ [F^ j]q{G,w) is determined by freqg.(w). 

Case 2: {/c + 1} 11,-'. This case is much easier to handle. Pick any j £ [k] such that 
n^/(j) = n^'(/c + 1). Then there is only one v € Vq such that typeQ{w,v) = t' (namely, 

Wj). 

Then X{t' , f ,t, f) = 1 if and only if for all F' G Conn^^;^, f'p, = fp, where F G Conn^ is 
the graph obtained by identifying the vertex labelled k + 1 with the vertex labelled j, and 
labelling this new vertex j. Otherwise A(r', /', r, /) = 0. 

This completes the definition of our desired function A. □ 



8 The Distribution of Labelled Subgraph Frequencies mod q 

In this section, we prove Theorem l6.12l As in Sectional the proof will be via an intermediate 
theorem (Theorem I8.2p that proves the equidistribution of the number of copies of labelled 
subgraphs in G{n,p). 

8.1 Equidistribution of labelled subgraph copies 

First, we gather some simple observations about injective homomorphisms from label- 
connected graphs for later use (the proofs are simple and are omitted). 
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Proposition 8.1 (Simple but delicate observations about label-connected graphs) 
Let F, F' G Conn^. Let G be a graph and let w £ Vq with all {wi)i^j distinct. 

L If E £Cop{F,{G,w)), the = l^^i^l. 

2. IfF^ F', we have Cop(F, (G, w)) n Cop(F', (G, w)) = 0. 

3. Let xit ■ ■ ,Xr £ lnj(-F, (G,w)) be such that for any distinct € [r], Xj(yF\^{F))ri 
Xf{VF\C{F)) = 0. Let X G lnj(F', (G, w)). Suppose x{Ef') <Z {UjXjiEp)). Then 
there is a j [r] such that x{Ef') ^ XjiEp)- 

We can now state and prove an equidistribution theorem for the number of copies of labelled 
subgraphs in a conditioned random graph. Theorem 16.121 will follow from this. 

Theorem 8.2 Let A be a graph with Va ^ [n] and \Va\ < n' . Let w = {wi, . . . ,Wk) G 
with wi, . . . ,Wk distinct. Let ui, . . .Ug S Va\ {wi : i £ 1} be distinct. Let Fi, . . . ,Fi be 
distinct k-labelled label- connected graphs, with 1 < \Ep.\ < d. Let Hi, . . . , H£' be distinct 
k + l-labelled label- connected graphs dependent on label k + 1, with 1 < l-Ej^J ^ d- 

Let G E G{n,p \ Va,Ea). Then the distribution of 

ii{Ei)q{G,w)),^[i],{{Hi>)g{G,w,Uj>)),,^[(,,]j,(,[,]) 
on Zg+*^' is 2~^9'P'''("~"')+(^+^''')'°S'?-c/ose to uniform in statistical distance. 

Proof By the Vazirani XOR lemma (Lemma l3.4p . it suffices to show that for any nonzero 

(c,c') eZ^x we have |E [lj^] \ < 2-^9.P,<i('^-'^')^ where 

R := ^q(F,),(G,w) c,^,{H,),{G,^,Uj,) 
and w E C is a primitive (j'^'^-root of unity. 

We will show this by appealing to Lemma [331 Let "t- = (2) ~ (2)- ^ ^ {0, l}^'^') be 
the random variable where, for each e G (^2^)) Zg = 1 if and only if edge e is present in G. 
Thus, independently for each e € ('2^) \ (^), Pr[ze = 1] = P, while for e G (^^)) the value 
of is either identically 1 or identically (depending on whether e G Ea or not). 

We may now express R in terms of the z^. We have, 
R=Y,c^{E^),{G,w)+ Yl E 

ieli] i'G[£']i'GM 

= Y ^^H^^^ Y '^'eH^^^ 
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where Tx C 2('2') is the set UieMxiT^o Cop(Fi, (K„, w)), T-i is the set Ui'g[£'] j'gH:c', ,,^o Cop(ifi/, (K„, w, Uj')), 
for each E G J^i, ce = Ci where i G [^] is such that E E Cop(Fj, (K^, w)) (note that by Propo- 
sitionOthere is exactly one such i), and similarly, for E G T2, c'^ = Ei'G[^'],i'GW:£;GCop(H,,,{K„,w,«y) 4i" 
Thus if E is such that there is a unique € [^'] x [s] for which £^ € Cop(i?i', (K„, w, Uj')) 

and c^, 7^ 0, then c'^ / 0. 

Let Q{Z) € ^q^Zi], where Z = {Ze)^^^in]-^<^^VAy be the polynomial 

^ CE Yl n ^ 4 JJ Ze 



Let2€{0,l}('^')\(^") be the random variable z restricted to the coordinates indexed by 
('2') \ {^2) (thus each coordinate of z independently equals 1 with probability p). Then 
R = Q(z). We wish to show that 



E 



UJ 



Q{z) 



< 2' 



j(n-n') 



(9) 



We do this by demonstrating that the polynomial Q{Z) satisfies the hypotheses of Lemma r3.31 

Let dl = maxj:ci7^o l-E-Fi I- Let = maxj/j/.c/ ^(j\EH^,\- We take cases depending on 
whether < or di > ^2- 

Case 1: Suppose < d^. Let igjin be such that c', ., / and |£'/f,, [ = d^. Then Q(Z) 

may be written as c'^ JleG-E + where T = {E G J-'2 ■ E D (^) = 0} and 

deg(Q') < dl 

Let xii X2, • • • J Xr G Injfffj/ , (K„, w, li,/ )) be a collection of homomorphisms such that: 
L For all j G [r], we have XjiVn^, \ ^H,, )) C [n] \ Va- 

2. For all distinct j,j' G [r], we have \ CiH.)) D XfiVn^, \ CiH.)) = 0. 

Such a collection can be chosen greedily so that r = 17 (^^^^). Let E'j G Cop(ffjj^, (K„,w,iij/^)) 
be given by XjiEn., )• Let £" be the family of sets {E'l, . . . , Er} Q T . We observe the fol- 
lowing properties of the Ef 

1. For each j G [r], \Ej \ = d\ (since Xj is injective and w\^. . . ,Wk,Uj^ are distinct). 

2. For each j G [r], c^^, / 0. This is because there is a unique (namely (io,Jo)) 
for which Cj/j/ 7^ and Ej G Cop(i?j', (K„, vi^, Uj')). Indeed, if j' 7^ jg, then each E* G 
Cop(i/i',(K „,w,Uj')) has some element incident on Uji (while Ej does not). On the 
other hand, if j' = Jq and i' 7^ i'q, then Pr op osition 18 . 1 1 implies that Cop(ifj/, (K„, w, Uji))n 
Cop(i?i;^,(K„,w,'Uj/)) = 0. 

3. For distinct j,j' G [r], Ej Pi £'jv = (by choice of the Xj)- 
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4. For any S G \ £, \S (1 {[JjEj)\ < dg- To see this, take any S & J^\£ and suppose 
\Sr\ iUjEj)\ > dl- Let i' G [f],/ G [s] be such that S G Cop(Fi/, (K„, w, lij/)). Let 
X G lnj(//j/, (K„, w, Uj/)) with xi^^H^,) = 'S'- By choice of dg) know that IS*! < d^- 
Therefore, the only way that 15" fl {\JjEj)\ can be > is if (a) l^l = d^, and (b) 
S n (UjEj) = S, or in other words, S C (UjEj). Since i^j/ is dependent on label 
k + 1, we know that S has some element incident on vertex Uj', and thus (b) forces 
j' = Jq (otherwise no Ej is incident on uj'). Now by Proposition 18 .11 this implies that 
S C Ej for some j. But since = we have S = Ej, contradicting our choice of 
S. Therefore, \S n {UjEj)\ < for any 5 G \ f . 

It now follows that Q{Zi),J^ and £ satisfy the hypothesis of Lemma 13.31 Consequently, 
(noting that < d) Equation Q follows, completing the proof in Case 1. 

Case 2: Suppose d^ > Let zq be such that ^ and l-Ej?.^ I = ^i- Then Q{Z) may be 

written as J2e(^A^e + c'^) DeGi? + (5'(Z), where = {£; G J^i U F2 : n (^^) = 0} and 
deg(g') < dl 

Let Xi) X2) • • • , Xr € lrij(Fjp, (K„, w)) be a collection of homomorphisms such that: 
L For all j G [r], we have \ C{Fi,)) C [n] \ Va- 

2. For ah distinct G [r], we have XjiVp,, \ ^Fi,)) n XjK^^',, \ ^F,,)) = 0. 

Such a collection can be chosen greedily so that r = 17( ""^" ). Let E'j G Cop(Fj(,, (K„, w)) be 
given by XjiEpi^)- Let £" be the family of sets {£'1, . . . ,Er} Q T. We observe the following 
properties of the Ej : 

1. For each j G [r], \Ej \ = d^ (since Xj is injective and wi, . . . ,Wk are distinct). 

2. For each j G [r], c^;. + Cp, / 0. This is because c^. = Ci„ / and for any 

Ej Cop{Hii, (K„, w,Mj/)) (and so c'^;, = 0). To see the latter claim, note that each 
E* G Cop(Fi/,(K n,w,?i,')) has an element incident on Uj' (which Ej does not). 

3. For distinct G [r], Ej (1 Ej/ = (by choice of the Xj)- 

4. For any S G \£, \S (1 {[JjEj)\ < d^. To see this, take any S £ J^\£ and suppose 

\sn{UjE,)\>dl. 

(a) If 5 G J^i, then let i G [i] be such that S G Cop(Fj, (K„, v^^)). Let x ^ 
lnj(Fi, (K„,w)) with x{EfJ = S. We know that \S\ < d*^. Therefore, the only 
way that \Sn([JjEj)\ can be > dl is if (1) \S\ = dl, and (2) Sn{UjEj) = S, or in 
other words, S C (UjEj). However, by Proposition 18.11 this implies that S C Ej 
for some j. But since \Ej \ = \S\, we have S = Ej, contradicting our choice of S. 

(b) If 5 G J^2, then let i' G [£'],j' G [s] be such that S G Cop(i/j/, (K„, w,Mj/)). Let 
X G lnj(i?j/, (K„, w,^^/)) with x{Eh^,) = S. We know that IS"! < < Now 
S has an element incident on Uj'. On the other hand none of the Ej have any 
edges incident on Uj'. Therefore \S (1 (UjEj)] < \S\ < d^. 
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Therefore, |5 n {UjEj)\ < for any 5 G JT \ 

It now follows that Q(Z),J^ and £ satisfy the hypothesis of Lemma 13.31 Consequently, 
(noting that < d) Equation Q follows, completing the proof in Case 2. □ 



8.2 Proof of Theorem [6A2] 

Theorem 16.121 (restated) Let a >b be positive integers. Let A be a graph with Va ^ [n] 
and \Va\ < ri' < n/2. Let G G G{n,p \ Va,Ea)- Let w = {wi, . . . ,Wk) G V^, and let 
ui,...,Us G Va \ {wi, . . . jWk} be distinct. Let r = type(^(w) and let Ti = type(j(w, Uj) 
(note that t,ti, . . . ,Ts are already determined by Ea). Let f denote the random variable 
freqg(w). Let fi denote the random variable freqQ(w, Uj). 

Then, there exists a constant p = p{a,q,p) > 0, such that if s < p ■ n, then the distribu- 
tion o/ (/, /i, . . . , fs) over FFreq„(T, [k],a) x Y\- FFreq„(Tj, [k + l],b) is 2~^^"'^ -close to the 
distribution of {h, hi, ... , hg) generated as follows: 

1. h is picked uniformly at random from FFreq„(r, [A;], a). 

2. For each i, each hi is picked independently and uniformly from the set of all f G 
FFreq„(rj, [k + such that (ri, f) extends {T,h). 

Proof Let v = w/LIt-. Let Fi, . . . , be an enumeration of the elements of Connfj^. 

Let n' G Partitions([A; + 1]) equal U {{k + 1}}. Notice that for each i G [s], n^, = W . Let 
Hi, . . . , Hi' be an enumeration of those elements of Conn^/ that are dependent on label i*. 

By Theorem 18.21 and the hypothesis on s for a suitable constant p, the distribution of 

{g,g^,..., g') = (((Fi)g(G, v))^^^^] , {{Hi,){G, v, ^ii'))i'G[£'] j'ew) 

is 2~^("') close to uniform over Z^"'"^''^. Given the vector {g, g^ , . . . ,g^), we may compute the 
vector (/, /i, . . . , fs) as follows: 

1. For F = Ki([A;]), we have fF = n— \IIt\- 

2. For all other F G Conn^, let i G [£] be such that F/Ur = F^. Then fF = 9i- aut(Fi). 

3. For H G Conn^_^i dependent on label k + I, let i' G [£'] be such that H/{U') ^ H[. 
Then for each / G [s], {fj')H = gi ■ aut(i/j/). 

4. For H G Con r\\^i not dependent on label k -\- 1 and for any j' G [s], there is a unique 
setting of {fj')H (given the settings above) that is consistent with the fact that {tj, fj) 
extends {T,f). This follows from Lemma |6. 91 

This implies the desired claim about the distribution of (/, /i, . . . , /c,). □ 
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9 Concluding Remarks 



The results presented here constitute the first systematic investigation of the asymptotic 
probabilities of properties expressible in first-order logic with counting quantifiers. More- 
over, these results have been established by combining, for the first time, algebraic methods 
related to multivariate polynomials over finite fields with the method of quantifier elimina- 
tion from mathematical logic. 

We conclude with two open problems: 

1. What is the complexity of computing the numbers ao, • • • in Theorem 12.11 .'' We 
know that it is PSPACE-hard to compute these numbers (it is already PSPACE-hard 
to tell if the asymptotic probability of a FO sentence is or 1). Our proof shows that 
they may be computed in time 2^ of height proportional to the quantifier depth of 
the formula. It is likely that a more careful analysis of our approximation of FO[l\/lodq] 
by polynomials can yield better upper bounds. 

2. Is there a modular convergence law for FO[Modm] for arbitrary m? The same obstacles 
that prevent the Razborov-Smolensky approach from generalizing to ACO[Mode] im- 
pede us. Perhaps an answer to the above question will give some hints for ACO[Mode]? 
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